Transformer-based models have gained large popularity and demonstrated promising results in long-term time-series forecasting in recent years. In addition to learning attention in time domain, recent works also explore learning attention in frequency domains (e.g., Fourier domain, wavelet domain), given that seasonal patterns can be better captured in these domains. In this work, we seek to understand the relationships between attention models in different time and frequency domains. Theoretically, we show that attention models in different domains are equivalent under linear conditions (i.e., linear kernel to attention scores). Empirically, we analyze how attention models of different domains show different behaviors through various synthetic experiments with seasonality, trend and noise, with emphasis on the role of softmax operation therein. Both these theoretical and empirical analyses motivate us to propose a new method: TDformer (Trend Decomposition Transformer), that first applies seasonal-trend decomposition, and then additively combines an MLP which predicts the trend component with Fourier attention which predicts the seasonal component to obtain the final prediction. Extensive experiments on benchmark time-series forecasting datasets demonstrate that TDformer achieves state-of-the-art performance against existing attention-based models.

Boundary conditions (BCs) are important groups of physics-enforced constraints that are necessary for solutions of Partial Differential Equations (PDEs) to satisfy at specific spatial locations. These constraints carry important physical meaning, and guarantee the existence and the uniqueness of the PDE solution. Current neural-network based approaches that aim to solve PDEs rely only on training data to help the model learn BCs implicitly. There is no guarantee of BC satisfaction by these models during evaluation. In this work, we propose Boundary enforcing Operator Network (BOON) that enables the BC satisfaction of neural operators by making structural changes to the operator kernel. We provide our refinement procedure, and demonstrate the satisfaction of physics-based BCs, e.g. Dirichlet, Neumann, and periodic by the solutions obtained by BOON. Numerical experiments based on multiple PDEs with a wide variety of applications indicate that the proposed approach ensures satisfaction of BCs, and leads to more accurate solutions over the entire domain. The proposed correction method exhibits a (2X-20X) improvement over a given operator model in relative $L^2$ error (0.000084 relative $L^2$ error for Burgers' equation).

最近，深度神经网络在时间序列的预测中越来越受欢迎。他们成功的主要原因是他们有效捕获多个相关时间序列的复杂时间动态的能力。这些深度预测者的优势才开始在有足够数量的数据的情况下开始出现。这对实践中的典型预测问题提出了挑战，在实践中，每个时间序列的时间序列或观察值有限，或者两者兼而有之。为了应对这些数据稀缺问题，我们提出了一个新颖的域适应框架，域适应预报员（DAF）。 DAF利用具有丰富数据样本（源）的相关领域的统计强度，以通过有限的数据（目标）提高感兴趣域的性能。特别是，我们使用基于注意力的共享模块，该模块与跨域跨域和私人模块的域歧视器一起使用。我们同时诱导域不变的潜在特征（查询和密钥）和重新培训特定特征（值），以使源和目标域上的预报员的联合训练。一个主要的见解是，我们对齐密钥的设计使目标域即使具有不同的特征也可以利用源时间序列。对各个领域的广泛实验表明，我们提出的方法在合成和现实世界数据集上优于最先进的基准，而消融研究验证了我们的设计选择的有效性。

我们考虑使用图形结构的概率预测问题，顶点的动态取决于其本地连接结构。我们提出了Gopher，一种方法，该方法将图形神经网络的感应偏差与神经杂物核对捕获了我们概率预测的内在局部连续动态。通过与基线模型进行比较，我们研究了这两个归纳偏差的好处，从而有助于解开每个益处的基线模型。我们发现，捕获图形结构对于准确的域概率预测和更具样本的高效模型至关重要。令人惊讶的是，我们的实验表明，尽管反映了真正的概率动态，但连续时间进化电感偏差很少没有利益。

定量回归是一种有效的技术，可以量化不确定性，符合挑战的潜在分布，并且通常通过在多个分位数水平上的联合学习提供完全概率预测。然而，这些关节分位数回归的常见缺点是\ textit {stantile交叉}，其违反了条件分位式函数的理想单调属性。在这项工作中，我们提出了增量（样条曲线）量子函数I（S）QF，灵活和有效的无分布定量位估计框架，其解决了与简单的神经网络层的定量交叉。此外，I（s）QF Inter /外推预测与底层训练不同的任意定量水平。配备了对I（S）QF表示的连续排名概率得分的分析评估，我们将方法应用于基于NN的时间系列预测案例，其中尤其是昂达训练的分位数的昂贵重新培训成本的节省重大。我们还提供了在序列到序列设置下我们提出的方法的泛化误差分析。最后，广泛的实验证明了在其他基线上提高了一致性和准确性误差。

基于预测方法的深度学习已成为时间序列预测或预测的许多应用中的首选方法，通常通常优于其他方法。因此，在过去的几年中，这些方法现在在大规模的工业预测应用中无处不在，并且一直在预测竞赛（例如M4和M5）中排名最佳。这种实践上的成功进一步提高了学术兴趣，以理解和改善深厚的预测方法。在本文中，我们提供了该领域的介绍和概述：我们为深入预测的重要构建块提出了一定深度的深入预测；随后，我们使用这些构建块，调查了最近的深度预测文献的广度。

There are multiple scales of abstraction from which we can describe the same image, depending on whether we are focusing on fine-grained details or a more global attribute of the image. In brain mapping, learning to automatically parse images to build representations of both small-scale features (e.g., the presence of cells or blood vessels) and global properties of an image (e.g., which brain region the image comes from) is a crucial and open challenge. However, most existing datasets and benchmarks for neuroanatomy consider only a single downstream task at a time. To bridge this gap, we introduce a new dataset, annotations, and multiple downstream tasks that provide diverse ways to readout information about brain structure and architecture from the same image. Our multi-task neuroimaging benchmark (MTNeuro) is built on volumetric, micrometer-resolution X-ray microtomography images spanning a large thalamocortical section of mouse brain, encompassing multiple cortical and subcortical regions. We generated a number of different prediction challenges and evaluated several supervised and self-supervised models for brain-region prediction and pixel-level semantic segmentation of microstructures. Our experiments not only highlight the rich heterogeneity of this dataset, but also provide insights into how self-supervised approaches can be used to learn representations that capture multiple attributes of a single image and perform well on a variety of downstream tasks. Datasets, code, and pre-trained baseline models are provided at: https://mtneuro.github.io/ .

The purpose of this work was to tackle practical issues which arise when using a tendon-driven robotic manipulator with a long, passive, flexible proximal section in medical applications. A separable robot which overcomes difficulties in actuation and sterilization is introduced, in which the body containing the electronics is reusable and the remainder is disposable. A control input which resolves the redundancy in the kinematics and a physical interpretation of this redundancy are provided. The effect of a static change in the proximal section angle on bending angle error was explored under four testing conditions for a sinusoidal input. Bending angle error increased for increasing proximal section angle for all testing conditions with an average error reduction of 41.48% for retension, 4.28% for hysteresis, and 52.35% for re-tension + hysteresis compensation relative to the baseline case. Two major sources of error in tracking the bending angle were identified: time delay from hysteresis and DC offset from the proximal section angle. Examination of these error sources revealed that the simple hysteresis compensation was most effective for removing time delay and re-tension compensation for removing DC offset, which was the primary source of increasing error. The re-tension compensation was also tested for dynamic changes in the proximal section and reduced error in the final configuration of the tip by 89.14% relative to the baseline case.

Compliance in actuation has been exploited to generate highly dynamic maneuvers such as throwing that take advantage of the potential energy stored in joint springs. However, the energy storage and release could not be well-timed yet. On the contrary, for multi-link systems, the natural system dynamics might even work against the actual goal. With the introduction of variable stiffness actuators, this problem has been partially addressed. With a suitable optimal control strategy, the approximate decoupling of the motor from the link can be achieved to maximize the energy transfer into the distal link prior to launch. However, such continuous stiffness variation is complex and typically leads to oscillatory swing-up motions instead of clear launch sequences. To circumvent this issue, we investigate decoupling for speed maximization with a dedicated novel actuator concept denoted Bi-Stiffness Actuation. With this, it is possible to fully decouple the link from the joint mechanism by a switch-and-hold clutch and simultaneously keep the elastic energy stored. We show that with this novel paradigm, it is not only possible to reach the same optimal performance as with power-equivalent variable stiffness actuation, but even directly control the energy transfer timing. This is a major step forward compared to previous optimal control approaches, which rely on optimizing the full time-series control input.

The previous fine-grained datasets mainly focus on classification and are often captured in a controlled setup, with the camera focusing on the objects. We introduce the first Fine-Grained Vehicle Detection (FGVD) dataset in the wild, captured from a moving camera mounted on a car. It contains 5502 scene images with 210 unique fine-grained labels of multiple vehicle types organized in a three-level hierarchy. While previous classification datasets also include makes for different kinds of cars, the FGVD dataset introduces new class labels for categorizing two-wheelers, autorickshaws, and trucks. The FGVD dataset is challenging as it has vehicles in complex traffic scenarios with intra-class and inter-class variations in types, scale, pose, occlusion, and lighting conditions. The current object detectors like yolov5 and faster RCNN perform poorly on our dataset due to a lack of hierarchical modeling. Along with providing baseline results for existing object detectors on FGVD Dataset, we also present the results of a combination of an existing detector and the recent Hierarchical Residual Network (HRN) classifier for the FGVD task. Finally, we show that FGVD vehicle images are the most challenging to classify among the fine-grained datasets.